Repository navigation
Phase 5: ADR 0050 and a Python MCP client verification - #1330
neuromechanist merged 4 commits into
Conversation
ADR 0050 formalizes the rule two phases learned the hard way: no WebAssembly in the Worker bundle, ever, for any package this server depends on. Both failures (numcodecs instantiating WASM from a runtime fetch, and hyparquet-compressors compiling hysnappy's WASM at module load and so crashing isolate startup for every request) were invisible to bun test and caught only under real workerd, which is why the smoke script is the gate rather than an optimization. It could not be filed until ADR 0049 landed on dev, since the index test enforces gapless numbering; the content staged in the design doc is now a pointer to the ADR. scripts/mcp/verify_streamable_http.py drives a target host with the official Python MCP SDK: the negotiated revision, all six tools with their cache hint, then a chain from search through describe and list_recordings into the three recording tools and both read_window modes, against a recording it discovers rather than one hardcoded here. mcp-smoke.sh checks our own framing with curl and says nothing about whether a real SDK can talk to us, and OSA speaks this SDK, so this is the client that has to work. It is also where the docs page will copy its examples from. Its SDK usage was validated against a local MCPServer harness rather than discovered during a cutover, which caught two things worth having: any check failure inside the client's task group came back as a bare "ExceptionGroup: unhandled errors" with none of the message that explains it, and an over-cap refusal can arrive either as a tool error or as a raised protocol error, so the check accepts both. Design doc records the verified SDK version and the two assumptions in #1297 that turned out to be wrong: there is no For-agents guide to extend, and OSA has no MCP runtime at all while its two existing NEMAR tools call an endpoint that now 404s.
test/adr-index.unit.test.ts validates gapless numbering, README index agreement, and superseded-target existence. Nothing in the path filters matched .context/**, so it never ran when those files changed: #1292 and #1330 both showed a fully green check list with the one test that could have failed skipped, and both were verified by hand instead. Same shape as the shared/** gap this file already documents, same fix. Wired into unit-pure, with a self-reference so editing the filter re-runs it.
📦 Package PreviewYour package is ready to test! bun install -g nemar-cli@PR1330Version: 🤖 Updated on each push. |
Review of scripts/mcp/verify_streamable_http.py found the script could not connect at all and could report a pass it had not earned. Fixed: - The default URL omitted the /mcp transport path. GET / is a descriptor and POST / answers Not Found, which during a cutover reads like an unprovisioned hostname. A bare host is now normalized and reported. - The over-cap probe wrapped its call in `except Exception`, so a 429 or a 502 became a PASS. Removed: both a tool error and a schema rejection arrive as is_error, so an exception there is a transport failure. - The probe sat exactly AT all three caps, which the schema accepts (`>`). Now one second over the duration cap with one channel, so exactly one cap is crossed, and the refusal text must name it. - The envelope check tested 3 of 14 required fields. Now all 14, plus non-null where the contract forbids null, and required only where the contract makes it required: it is .optional() on three tools, so failing hard everywhere reported missing provenance against a valid legacy index. - The subject was `results[0]`, now scanned for a v3 index with a usable group. has_zarr means converted, not v3, so a legacy hit is legitimate. - Read `results` only; `datasets` is catalog.json's field. - Notes are counted and repeated in the closing line, so a degraded run cannot look clean. Adds the wire-level test the script's cap assertion depends on: an over-cap taste answers isError with a text block naming the cap and the recipe remedy. Validated against a local MCPServer stand-in with twelve fault injections; all ten defects fail with the right message and the two honest degradations exit 0 with a note.
An accuracy review of ADR 0050 and the design doc's phase 5 entry found four wrong claims, two of which would misdirect future work. All verified before changing, two by re-running the experiment. ADR 0050 said there is no flag permitting dynamic WASM compilation, so the restriction was the platform's and not configurable. Inverted. Measured under wrangler dev --local: compiling at MODULE LOAD is permitted by allow_eval_during_startup, which is the default from compatibility_date 2025-06-01, so the phase 3 shape is blocked only because this worker pins 2024-12-01. Compiling inside a request stays disallowed under every combination tested. So the decision survives but gains a real consequence: a routine compatibility_date bump silently removes the guard that caught the phase 3 failure, which is why the rule is written down and smoke-gated rather than left to the runtime. The flag's docs never mention WebAssembly, so the ADR now says the behavior is measured, not documented. ADR 0050 also framed both failures as dependencies that broke the server with a green suite. Neither was ever in backend/package.json: numcodecs was the spike's path (a), and hyparquet-compressors was removed before merge. The lesson holds, the history does not, and an ADR outranks the design doc. Design doc: the For-agents guide #1297 names does exist, published at docs platform/for-agents.md since 2026-09-03, so the docs work is a section on that page and #1297's premise was right. develop/zarr-contract.md is gone, split into six pages under platform/zarr/. Both errors came from reading a stale local checkout instead of origin/main. Also: #1324 has three non-handler exports, not two strings; the fixtures are real bytes where a real store exercises the path and synthetic only where the catalog has no such shape; the subpath-import option is named; and the custom-domain zone-edit caveat is moved next to the sequencing it gates.
Review rounds addressedTwo reviews ran on this PR. Both found real defects; everything is fixed in Round 1, the verification script. The headline finding is that the script Round 2, factual accuracy of ADR 0050 and the design doc. Four wrong claims, The ADR's central platform claim was inverted. It said no flag permits dynamic
So the phase 3 shape is configurable and blocked only because we pin The ADR also framed both incidents as dependencies that broke the server with a The design doc aimed the docs work at a page that does not exist and away from What was verified rather than asserted
Gates: 254 MCP tests, ADR index 8/8, typecheck and biome clean, |
1de3452
into
feature/issue-1065-epic-mcp-server
Summary
Part A of phase 5, the in-repo half. The rest of the phase (docs page, OSA runtime, release) is blocked on deployment and follows in order.
Closes nothing yet; #1297 stays open until the docs and OSA halves land.
Part of epic #1065.
ADR 0050: no WebAssembly in the Worker bundle
Formalizes the rule two phases learned the hard way.
numcodecsinstantiated WASM from a runtimefetch();hyparquet-compressorscompiledhysnappy's WASM at MODULE LOAD and so crashed isolate startup for every request, not justget_eventscalls. Both were invisible tobun test, which is whybackend/scripts/mcp-smoke.shunder real workerd is the gate rather than a nicety.It could not be filed earlier: the ADR index test enforces gapless numbering and 0049 was still open in #1292. That merged, so the content staged in the design doc is now a pointer to the real ADR.
The ADR is honest about the gate's own gap: the smoke script drives a throwaway entry module because
wrangler dev --localcannot start the realindex.ts(#1324), so it proves the MCP sub-app's dependency graph starts under workerd, not that the whole worker does.scripts/mcp/verify_streamable_http.pyDrives a target host with the official Python MCP SDK: the negotiated protocol revision, all six tools with the
tools/listcache hint, then a chain fromsearch_datasetsthroughdescribe_datasetandlist_recordingsintoget_events,render_overviewand bothread_windowmodes, against a recording it discovers rather than one hardcoded here.Three reasons it exists, none covered by the TypeScript tests:
mcp-smoke.shdrives our own framing withcurl. It says nothing about whether a real MCP SDK, doing its own handshake, can talk to us. OSA speaks this SDK, so this is the client that has to work.devand against production after the release.Its SDK usage was validated against a local
MCPServerharness before first use, rather than discovered during a cutover. That caught two real problems:ERROR ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)with none of the message explaining what failed. Now unwrapped, so a failure readsFAIL render_overview: no image content block on the result.read_windowrefusal can arrive either as a tool error (is_error) or as a raised protocol error, depending on whether the input schema or the tool rejects it. The check accepts both, since both are refusals and asserting one would report a false failure against the other.Verified API surface (mcp 2.2.0, the current release, checked by introspection rather than recalled): high-level
Client("<url>")speaks Streamable HTTP and negotiated2026-07-28;FastMCPwas renamedMCPServerin 2.x; Python-side result fields are snake_case (structured_content) while the wire keepsstructuredContent.Two assumptions in #1297 that turned out to be wrong
Both recorded in the design doc, both change what the remaining parts do:
nemarOrg/docsornemarOrg/website. The docs work is a new page, modeled ondocs's owndevelop/zarr-contract.md.mcpdependency, no client code, nothing readsmcp_serversoutside the config model whose docstring still says "(Phase 2)". And its two existing NEMAR tools callnemar.org/api/dataexplorer/datapipeline/..., which returns 404 since the legacy site was retired. So this server replaces them rather than supplementing them. Per the owner's decision, that runtime gets built in this phase.Why the rest is not in this PR
mcp.nemar.organdmcp-test.nemar.orgarecustom_domain = trueroutes, so neither hostname exists until a deploy provisions it. Neither resolves today. Verification is impossible until the epic reachesdev; production waits on the release. Writing docs or OSA wiring against an unverified server would mean describing behavior nobody has seen.Test plan
bun test test/adr-index.unit.test.ts8 of 8, including the gapless check that previously blocked filing 0050.bun run typecheckclean.MCPServerharness over Streamable HTTP; the full chain runs against a live host in Part C.